A New Statistical Model Combining Shape and Spherical Harmonics Illumination for Face Reconstruction

This paper develops a new statistical model that can be used to recover the 3D face shape from a single image taken under arbitrary lighting conditions. The proposed statistical model combines shape and image intensity information. The latter is represented by a spherical harmonics (SH) space. Given a training data set of aligned height maps of faces and their corresponding albedos, The input image is projected into the space spanned by the SH basis images of each member in the data set. The combined statistical model is obtained by performing the PCA on both the SH projections and the height maps. Finally, the face is reconstructed by fitting the model to the input intensity image. In addition to the shape recovery, the proposed model can be used to recover the face albedo and to estimate the light direction. Experiments have been conducted to evaluate the performance of the proposed approach.